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Abstract — This paper addresses the problem of illumination 
effects on Face recognition and works for an approach to 
reduce their effect on recognition performance. More broadly, 
a simple and efficient preprocessing chain that eliminates most 
of the effects of changing illumination while still preserving the 
essential appearance details that are needed for recognition. 
Using local ternary patterns (LTP), a generalization of the 
local binary pattern (LBP) local texture descriptor that is more 
discriminant and less sensitive to noise in uniform regions. We 
also show that replacing comparisons based on local spatial 
histograms with a distance transform based similarity metric 
further improves the performance of LBP/LTP based face 
recognition; and Robustness is still improved by adding Kernel 
principal component analysis (PCA) feature extraction. 
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I. Introduction 

This paper focuses mainly on the issue of robustness to 
lighting variations. For example, a face verification system 
for a portable device should be able to verify a client at any 
time (day or night) and in any place (indoors or outdoors). 
Unfortunately, facial appearance depends strongly on the 
ambient lighting Traditional approaches for dealing with this 
issue can be broadly classified into three categories: 
appearance-based, normalization-based, and feature-based 
methods. In this paper, we propose an integrative framework 
that combines the strengths of all three of the above 
approaches. The overall process can be viewed as a pipeline 
consisting of image normalization, feature extraction, and 
subspace representation, as shown in Fig. 1. Each stage 
increases resistance to illumination 

variations and makes the information needed for recognition 
more manifest. The method centres on a rich set of robust 
visual features that is selected to capture as much as 
possible of the available information. A well-designed 
image preprocessing pipeline is prepended to further 
enhance robustness. In this paper, combinations of all the 
above approaches are used. The schematic representation of 
the whole framework is given as follows, which contains 
preprocessing stage, feature extraction stage and subspace 
representation stage for recognition. 
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Figure 2. Stages of our image preprocessing pipeline 

Gamma correction - An initial level of corrections are 
made for illumination deficiency. However this does not 
remove the influence of overall intensity gradients such as 
shading effects 

Difference of Gaussians (DoG) Filtering - This involves 
the subtraction of one blurred version of an original gray 
scale image from another less blurred version of the original. 
This enhances the fine details of the image. 

Masking - At this stage, irrelevant regions like of the 
images are masked out. 

Equalization of Variation (EoV) - At this stage, 
intensities of the image are rescaled. This step is required to 
standardize the intensity values for further processing. 



Figure 1 . Stages of Face Recognition Method 
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III. TEXTURE BASED FEATURE EXTRACTION 

A. Local Binary Patterns (LBP) 

LBP operator summarizes local gray-level structure. The 
operator takes a local neighborhood around each pixel, 
thresholds the pixels of the neighborhood at the value of the 
central pixel and uses the resulting binary-valued image 
patch as a local image descriptor. 



LBP's are resistant to lighting effects and are shown to 
have high discriminative power for texture classification [5]. 
However, they are sensitive. Given that many facial regions 
are relatively uniform, it is potentially useful to improve the 
robustness of the underlying descriptors in these areas. 

It was originally defined for 3x3 neighborhoods, giving 
8-bit integer LBP codes based on the eight pixels around the 
central one. Formally, the LBP operator takes the form 



LBP(Xc, Yc) = ]T TSiin - ic). 



■d) 



n=Q 



where in this case runs over the 8 neighbors of the central 
pixel , and are the gray-level values at and , and is 1 if and 
otherwise. The LBP encoding process is illustrated in Fig. 3. 
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Figure 3. Basic LBP operator 



B 



Local Ternary Patterns (LTP) 

LTP, which are derived from LBP, are also used. In this 
gray-levels in a zone of width +t around a center pixel are 
quantized to zero, ones above this are quantized to +1 and 
ones below it to -1. 

Thus obtained LBP and LTP are used as texture feature 
descriptors. 

LBPs have proven to be highly discriminative features 
for texture classification and they are resistant to lighting 
effects in the sense that they are invariant to monotonic gray- 
level transformations. Many facial regions are relatively 
uniform and it is legitimate to investigate whether the 
robustness of the features can be improved in these regions. 
This section extends LBP to 3-valued codes, LTP, in which 
gray-levels in a zone of width around are quantized to zero, 
ones above this are quantized to 1 and ones below it to 1, i.e., 
the indicator is replaced with a 3-valued function and the 
binary LBP code is replaced by a ternary LTP code. Here is a 
user-specified threshold — so LTP codes are more resistant to 
noise, but no longer strictly invariant to gray-level 
transformations. The LTP encoding procedure is illustrated 
in Fig. 4. Here the threshold was set to 5, so the tolerance 
interval is 



s\ujc,t) = { 



1 u>ic + t 
u-ic + t 
-1 u<ic + t 



(2) 
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Figure 4. Illustration of the basic LTP operator. 



When using LTP for visual matching, we could use 
valued codes, but the uniform pattern argument also applies 
in the ternary case. For simplicity, the experiments below use 
a coding scheme that splits each ternary pattern into its 
positive and negative halves as illustrated in Fig. 5, 
subsequently treating these as two separate channels of LBP 
descriptors for which separate histograms and similarity 
metrics are computed, combining the results only at the end 
of the computation. 
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Figure 5. Splitting an LTP code into Positive and Negative LBP codes. 

C. Distance transform based Similarity Measure 

The used similarity metric is an LBP based method for 
face recognition [6] that divides the face into a regular grid 
of cells and histograms the uniform LBP's within each cell, 
finally using nearest neighbor classification in the y2 
histogram distance for recognition: 



{pi - gO 
{pi - qi) 



(3) 



Here p,q are image region descriptors (histogram 
vectors), respectively 

However, subdividing the face into a regular grid seems 
somewhat arbitrary: the cells are not necessarily well aligned 
with facial features, and the partitioning is likely to cause 
both aliasing (due to abrupt spatial quantization of descriptor 
contributions) and loss of spatial resolution (as position 
within each grid cell is not coded). Given that the overall 
goal of coding is to provide illumination- and outlier -robust 
visual correspondence with some leeway for small spatial 
deviations due to misalignment, it seems more appropriate to 
use a Hausdorff- distance-like similarity metric that takes 
each LBP or LTP pixel code in image and tests whether a 
similar code appears at a nearby position in image , with a 
weighting that decreases smoothly with image distance. Such 
a scheme should be able to achieve discriminant appearance- 
based image matching with a well-controllable degree of 
spatial looseness. We can achieve this using distance 
transforms. Given a 2-D reference image, we find its image 
of LBP or LTP codes and transform this into a set of sparse 
binary images , one for each possible LBP or LTP code value 
(i.e., 59 images for uniform codes). Each specifies the pixel 
positions at which its particular LBP or LTP code value 
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appears. We then calculate the distance transform image of 
each . Each pixel of gives the distance to the nearest image 
pixel with code (2-D Euclidean distance is used in the 
experiments below). The distance or similarity metric from 
image to image is then 



D(X,Y)= ^w(d K x %mj)) 

pixels (i, /') of Y 



(4) 



Here, is the code value of pixel of image and is a user- 
defined function4 giving the penalty to include for a pixel at 
the given spatial distance from the nearest matching code in . 
In our experiments we tested both Gaussian similarity 
metrics and truncated linear distances. Their performance is 
similar, with truncated distances giving slightly better results 
overall. For 120x120 face images in which an iris or nostril 
has a radius of about six pixels and overall global face 
alignment is within a few pixels, our default parameter 
values were pixels and pixels. 




Figure 6. From left to right: a binary layer, its distance transform, and the 
truncated linear version of this. 

Fig. 6 shows an example of a binary layer and its distance 
transforms. For a given target the transform can be computed 
and mapped through in a preprocessing step, after which 
matching to any subsequent image takes (number of pixels) 
irrespective of the number of code values. 

IV. RESULTS 
Following is the results during the preprocessing stage: 
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Figure 7: Pre-processing stages: (a) Original Image, (b) After Gamma 
filtering (c) After Gaussian filtering and (d) After EoV (e) Before Pre- 
processing and (f) After Pre-processing 
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Figure 8: Experimental Results based on literature survey 

V. Abbreviations and Acronyms 

LTP - Local Ternary Patterns 

LBP - Local Binary Pattern 

PCA- Principal Component Analysis 

FRGC-204 - Face Recognition Grand Challenge Version 
2 Experiment 4 

FRVT - Face Recognition Vendor Test 

FRGC - Face Recognition Grand Challenge Trials 

MSR - Multiscale Retinex 

SQI - Self Quotient Image 

LTV - Logarithmic Total Variation 

GB - Gross and Brajovic 

HE - Histogram Equalization 

VI. CONCLUSION 

We have presented new methods for face recognition 
under uncontrolled lighting based on robust preprocessing 
and an extension of the LBP local texture descriptor. The 
main contributions are as follows: 

1) A simple, efficient image preprocessing chain whose 
practical recognition performance is comparable to or better 
than current (often much more complex) illumination 
normalization methods; 
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2) A rich descriptor for local texture called LTP that 
generalizes LBP while fragmenting less under noise in 
uniform regions: 

3) A distance transform based similarity metric that 
captures the local structure and geometric variations of 
LBP/LTP face images better than the simple grids of 
histograms that are currently used; and 

4) A heterogeneous feature fusion-based recognition 
framework that combines two popular feature sets — Gabor 
wavelets and LBP — with robust illumination normalization 
and a kernelized discriminative feature extraction method. 
The combination of these enhancements gives the state-of- 
the-art performance on three well-known large-scale face 
datasets that contain widely varying lighting conditions. 
Moreover, we empirically make a comprehensive analysis 
and comparison with several state-of-the-art illumination 
normalization methods on the large-scale FRGC-204 dataset, 
and investigate their connections with robust descriptors, 
recognition methods, and image quality. This provides new 
insights into the role of robust preprocessing methods played 
in dealing with difficult lighting conditions and thus being 
useful in the designation of new methods for robust face 
recognition. 
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